The performance gap between the processor and the memory system in computing systems has been steadily increasing. With every generation, processors are being clocked at higher rates, while memory systems have been unable to catch up to this exponential growth. The resultant performance gap has caused a major bottleneck for single-thread performance in modern day processors, as the memory system is generally unable to supply data to the processor core at the rate of execution of the processor.
Traditional means of reducing this performance gap include the use of cache memory of varying sizes and levels, which provides temporary storage for and quick access to frequently used data. Cache memory is conventionally managed by hardware, which attempts to cache the most frequently accessed data while purging older, unused data and fetching data from nearby memory locations, thus retaining the working set of the program in cache. Other approaches implement software-managed cache or “scratch pads” for better data management.
Although there are many advantages to using hardware-managed and software-managed cache systems, a common drawback of both is the difficulty in efficiently managing cache replacement (i.e., when to purge old data and replace it with new data). In particular, it is difficult to determine whether certain older data in cache may be purged to make room for newer data, or whether the older data needs to be kept longer in cache (i.e., may still be useful). Conventional schemes for assigning priority to data in cache rely solely programmer-specified hints or compile-time analysis, and thus are based on static determinations that do not account for run-time changes.
Thus, there is a need in the art for a method and apparatus for dynamic priority-based cache replacement.